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Abstract 

Consider an operator equation F(u) = in a real Hilbert space. The problem 
of solving this equation is ill-posed if the operator F'(u) is not boundedly invertible, 
and well-posed otherwise. A general method, dynamical systems method (DSM) for 
solving linear and nonlinear ill-posed problems in a Hilbert space is presented. This 
method consists of the construction of a nonlinear dynamical system, that is, a Cauchy 
problem, which has the following properties: 1) it has a global solution, 2) this solution 
tends to a limit as time tends to infinity, 3) the limit solves the original linear or non- 
linear problem. New convergence and discretization theorems are obtained. Examples 
of the applications of this approach are given. The method works for a wide range of 
wcll-poscd problems as well. 

1 Introduction 

This paper contains a recent development of the theory of DSM (dynamical systems 
method) earlier developed in papers [2]-[12]. DSM is a general method for solving op- 
erator equations, especially nonlinear, ill-posed, but also well-posed operator equations. 
The author hopes that DSM will demonstrate its practical efficiency and will allow one to 
solve ill-posed problems which cannot be solved by other methods. This paper is intended 
for a broad audience: the presentation is simplified considerably and is non-technical in 
its present form. Most of the results are presented in a new way. Some of the results 
and/or proofs are new (Theorems 2.1, 3.1, 3.2, 4.2, 6.2, 7.1, 8.1, Remarks 4.4, 4.5, and the 
discussion of the stopping rules) . We try to emphasize the basic ideas and methods of the 
proofs. 

What is the dynamical systems method (DSM) for solving operator equations? 
Consider an equation 

F(u) := B(u) - f = 0, feH, (1.1) 
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where B is a linear or nonlinear operator in a real Hilbert space H. Some of our results can 
be generalized to more general spaces, but these generalizations are not discussed here. 
Throughout the paper we assume that: 

sup \\F^{u)\\ <M jt j = 1,2, (1.2) 

u&B(u ,R) 

where B(uo, R) := {u : \ \u — uq || < R}, FW(u) is the Frechet derivative, and 

F(y) = 0, yeB(u ,R), (1.3) 

that is, we assume existence of a solution to (1.1), not necessarily uique globally. 

Assumptions (1.2) and (1.3) are our standard assumptions below, unless otherwise 
stated. Only for well-posed problems in Section 2 we do not assume existence of a solution, 
but prove it, and sometimes we can assume in these problems j = 1 in (1.2), rather than 
j = 2. In all the ill-posed problems we assume existence of the solution to (1.1). 

Let u denote derivative with respect to time. Consider the dynamical system ( the 
Cauchy problem ): 

u = $(t,u), u(0)=u , (1.4) 

where <!>(£, u) is locally Lipschitz with respect to u £ H and continuous with respect to 
t > 0: 

sup \\&(t,u)-&(t,v)\\ <c||«-t;||, c = c(R,u ,T) > 0. (1.5) 

u,veB(u ,R),te[0,T] 

One can relax "locally Lipschitz" assumption about $ (for example, use one-sided inequal- 
ities), but we do not discuss this point. Problem (1.4) has a unique local solution if (1.5) 
holds. The DSM for solving (1.1) consists of solving (1.4), where <3? is so chosen that the 
following three conditions hold: 

3u(t)Vt > 0; 3u(oo) := lim u(t); F(u(oo)) = 0. (1.6) 

t^oo 

Some of the basic results of this paper are the Theorems which provide the choices of $ 
for which (1.6) holds, and the technical tools (Theorems 4.1 and 6.1) basic for our proofs. 

Problem (1.1) with noisy data fg, \\fs — f\\ < 5, given in place of /, generates the 
problem: 

us = $s(t, us), us(0)=u , (1.7) 
The solution us to (1.7), calculated at t = t$, will have the property 

limJM^-yll =0. (1.8) 

The choice of tg with this property is called the stopping rule. One has usually lim^o *<5 = 

oo. 
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In Section 2 we discuss well-posed problems (1.1), that is, the problems for which 

sup Hli^u)]- 1 !! <mi, (1.9) 

u£B(u ,R) 

and in the other sections ill-posed problems (1.1), for which (1.9) fails, are discussed. 
The motivations for this work are: 

1) to develop a general method for solving operator equations, especially nonlinear and 
ill-posed, 

and 

2) to develop a general approach to constructing convergent iterative schemes for solv- 
ing these equations. 

If (1.6) holds, and if one constructs a convergent discretization scheme for solving 
Cauchy problem (1.4), then one gets a convergent iterative scheme for solving the original 
equation (1.1). 

2 Well-posed problems 

Consider (1.1), let (1.2) hold, and assume 

/•OO 

(F'(u)<S>(t,u),F( U )) <- gi (t)\\F(u)\\ a Vu£B(u ,R), / 9l dt = oo, (2.1) 

J o 

where g\ > is an integrable function, a > is a constant. Assume 

\\Ht,u)\\ < g 2 (t)\\F(u)\\, G B(uq, R), (2.2) 
where 52 > is such that 

G(t) :=g 2 (t)exp(- f 9l ds) G L 1 (R + ). (2.3) 
Jo 

Remark: Sometimes the assumption (2.2) can be used in the following modified form: 

\\Ht, U )\\<g 2 (t)\\F( U )\\ b VuGB, (2.2') 

where b > is a constant. The statement and proof of Theorem 2.1 can be easily adjusted 
to this assumption. 

Our first basic result is the folowing: 

Theorem 2.1. i) If (2.1)- (2.3) hold, and 

roc 

\\F(u )\\ / G(t)dt<R, a = 2, (2.4) 
J 
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then (1-4) has a global solution, (1.6) holds, (1.1) has a solution y = u(oo) G B(uq,R), 
and 



poo Pt 

\\u(t)-y\\<\\F(u )\\ G(x)dx, \\F(u(t))\\<\\F(u )\\exp(- 9l (x)dx). (2.5) 
Jt Jo 

ii) If (2.1)- (2. 3) hold, < a < 2, and 

\\F(u )\\ f g 2 ds<R, (2.6) 
Jo 

where T > is defined by the equation 

[ T gi ( S )ds = \\F(u )\\ 2 - a /(2-a), (2.7) 
Jo 

then (1-4) has a global solution, (1.6) holds, (1.1) has a solution y = u(oo) € B(uq,R), 
and u(t) = y for t >T. 

Hi) If (2.1)- (2.3) hold, a > 2, and 

/ g 2 (s)h(s)ds < R, (2.8) 
Jo 

where ^ 

[\\F(u )\\ 2 - a + (a-2) [ gi (s)ds]^ :=h(t), lim h(t) = 0, (2.9) 

then (1-4) has a global solution, (1.6) holds, (1.1) has a solution y = u(oo) £ B(uq,R), 
and 

/oo 
g 2 (s)h(s)ds -► (2.10) 

as t — > oo. 

Let us sketch the proof. 

Proof of Theorem 2.1. The assumptions about $ imply local existence and unique- 
ness of the solution u(t) to (1.4). To prove global existence of u, it is sufficient to prove 
a uniform with respect to t bound on ||u(i)||. Indeed, if the maximal interval of the ex- 
istence of u(t) is finite, say [0, T), and $(t,u) is locally Lipschitz with respect to u, then 
||ti(t)|| oo as t -> T. 

Assume a = 2. Let g(t) := ||F(n(t))||. Since H is real, one uses (1.4) and (2.1) to 
get gg = (F'(u)u,F) < —gi(t)g 2 , so g < —g\{t)g, and integrating one gets the second 
inequality (2.5), because g(0) = ||F(no)||. Using (2.2), (1.4) and the second inequality 
(2.5), one gets: 



\u{t) 



- u{s)\\ < g(0) f G(x)dx, G(x) := g 2 (x) exp(- [* 9l (z)dz). (2.5') 

is Jo 
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Because G G L l (R + ), it follows from (2.5') that the limit y : = lim^oo u{t) = u(oo) exists, 
and y G B by (2.4). From the second inequality (2.5) and the continuity of F one gets 
F(y) = 0, so y solves (1.1). Taking t — > oo and setting s = t in (2.5') yields the first 
inequality (2.5). The inclusion u(t) G 5 for all t > follows from (2.4) and (2.5'). The 
first part of Theorem 2.1 is proved. The proof of the other parts is similar. □ 

There are many applications of this theorem. We mention just a few, and assume that 
<7i = C\ = const > and <?2 = C2 = const > 0. 

Example 1. Continuous Newton-type method (Gavurin, (1958)): 

<I> = -[F'(u)]~ 1 F(u). Assume that (1.9) holds, then c\ = 1,C2 = mi, (2.4) takes the 
form (*) mi(i?)||F(uo)|| < R, and (*) implies that (1.4) has a global solution, (1.6) and 
(2.5) hold, and (1.1) has a solution in B(uq,R). 

Example 2. Continuous simple iterations method: 

Let $ = —F, and assume F'(u) > c±(R) > for all u G B(u ,R). Then c 2 = 1, 
c\ = c\{R), (2.4) is: [ci(i?)] _1 ||F(«o)|| < and the conclusions of Example 1 hold. 
Example 3. Continuous gradient method: 

Let $ = -[F']*F, (1.2) and (1.9) hold, ci = m^ 2 , c 2 = Mi (12), (2.4) is (**) 
Mimi||.F(uo)|| ^ l^j an d (**) implies the conclusions of Example 1. 

Example 4. Continuous Gauss-Newton method: Let $ = - ([F']*F') 
(1.2) and (1.9) hold, ci = 1, c 2 = m 2 Mi, (2.4) is (***) Mim 2 | |F(u )| I < R, and (***) 
implies the conclusions of Example 1. 

Example 5. Continuous modified Newton method: Let $ = — [F' (uq)]" 1 F(u). 
Assume 1 1 [F'(uo)] _1 | | < m , and let (1.2) hold. Then c 2 = m . Choose 1? = (2M 2 m ) _1 , 
and ci = 0.5. Then (2.4) is 2m | |F(it )| | < (2M 2 mo) _1 , that is, Am\M 2 \ \F(u )\ \ < 1. 
Thus, if 4m 2 1 M2||l ? (uo)|| < 1, then the conclusions of Example 1 hold. 

Example 6. Descent methods. 

Let = — tf/^ h, where / = f{u{t)) is a differentiable functional / : H — > [0, 00), and 

/i is an element of H. From (1.4) one gets / = (f',u) = —f. Thus / = foe~ l , where 
/o := f(u ). Assume ||$|| < c 2 1 /| 6 , b > 0. Then ||«|| < c 2 |/o| b e~ b *. Therefore u(oo) does 
exist, /(u(oo)) = 0, and ||u(oo) — u(t)\\ < ce~ bt , c = const > 0. 

If h = /', and / = \\F(u)\\ 2 , then f(u) = 2[F']*(u)F(u), $ = -jfof, and (1.4) 
is a descent method. For this <D one has c\ = ^, and C2 = where mi is defined in 
(1.9). Condition (2.4) is: mi||F(«o)|| < R. If this inequality holds, then the conclusions 
of Example 1 hold. 

In Example 6 we have obtained some results from [1]. Our approach is more general 
than the one in [1], since the choices of / and h do not allow one, for example, to obtain 
$ used in Example 5. 
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3 Linear ill-posed problems 



We assume that (1.9) fails. Consider 

Au = f. (3.1) 

Let us denote by A) the folowing assumption: 

A): A is a linear, bounded operator in H, defined on all of H, the range R(A) is not 
closed, so (3.1) is an ill-posed problem, there is a y such that Ay = f, y _L N, where N is 
the null-space of A. 

Let B := A* A, q := By = A*f, A* is the adjoint of A. Every solution to (3.1) solves 

Bu = q, (3.2) 

and, if / = Ay, then every solution to (3.2) solves (3.1). Choose a continuous, monotoni- 
cally decaying to zero function e(t) > 0, on R + . 
Sometimes it is convenient to assume that 

lim (ee~ 2 ) = 0. (3.3) 

t— +oo 

For example, the functions e = c\{cq + t)~ b , < b < 1, where cq and c\ are positive 
constants, satisfy (3.3). There are many such functions. One can prove ([4], [8]) the 
following: 

Claim: Ife(t) > is a continuous monotonically decaying function onW + , lim^oo e(t) = 
0, and (3.3) holds, then 

/•oo 

/ eds = oo. (3.3') 
Jo 

In this Section we do not use assumption (3.3): in the proof of Theorem 3.1 one uses 
only the monotonicity of a continuous function e > and (3.3'). One can drop assumption 
(3.3'), but then convergence is proved in Theorem 3.1 to some element of N, not necessarily 
to the normal solution y, that is, to the solution orthogonal to N, or, which is the same, 
to the minimal norm solution to (3.1). However, (3.3) is used (in a slightly weaker form) 
in Section 4. 

Consider problems (1.4) and (1.7) with 

$ := -[Bu + e(t)u - q], $ 5 = -[Bu s + e(t)u s - q S ], (3.4) 

where \\q — q$\\ < \\A*\\5 := C5. Without loss of generality one may assume this C = 1, 
which we do in what follows. Our main result in Sec. 3, is Theorem 3.1, stated below. It 
yields the following: 

Conclusion: Given noisy data f$, every linear ill-posed problem (3.1) under the as- 
sumptions A) can be stably solved by the DSM. 

The result presented in Theorem 3.1 is essentially obtained in [8], but our proof is 
different and much shorter. 
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Theorem 3.1. Problem (1-4) with from (3.4) has a unique global solution u(t), 
(1.6) holds, andu(oo) = y. Problem (1.7) with $>s from (3.4), has a unique global solution 
us(t), and there exists t$, such that 

lim \ \us(ts) — 2/| | = 0. (3.5) 

5— >0 

This ts can be chosen, for example, as a root of the equation 

e{t)=5 b , be (0,1), (3.6) 

or of the eqation (3.6'), see below. 

Proof of Theorem 3.1. Linear equations (1.4) with bounded operators have unique 
global solutions. If $ = — [Bu + e{t)u — q], then the solution u to (1.4) is 

r\\B\\ rt 

u(t) = h- 1 (t)U(t)u + h~ 1 (t) exp(-U) e sX h(s)dsXdE x y, (3.7) 

Jo Jo 

where h(t) := exp(J^ e(s)ds) — ► oo as t — ► oo, E\ is the resolution of the identity cor- 
responding to the selfadjoint operator B, and U(t) := e~ tB is a nonexpansive operator, 
because B > 0. Actually, (3.7) can be used also when B is unbounded, \ \B\ \ = oo. 
Using L'Hospital's rule one checks that 



, Xj'e sX h(s)ds , Xe tx h(t) 
lim J ° tyttV = lim T7T771 ,\ , w , =1 VA > 0, 3.8 

t^oo e u /i(t t^oo Xe tx h{t) + e tx h{t)e{t) 



provided only that e(t) > and lim^oo e(t) = 0. From (3.7), (3.8), and the Lebesgue 
dominated convergence theorem, one gets u(oo) = y — Py, where P is the orthogonal 
projection operator on the null-space of B. Under our assumptions A), P = 0, so u{co) = 
y. If v(t) := \\u(t) — y\\, then lim^ 00 u(t) = 0. In general, the rate of convergence of 
v to zero can be arbitrarily slow for a suitably chosen /. Under an additional a priori 
assumption on / (for example, the source type assumptions), this rate can be estimated. 
Let us describe a method for deriving a stopping rule. One has: 

\\u 5 (t)-y\\ < \\u s (t)-u(t)\\ + v(t). (3.9) 

Since lim^oo v(t) = 0, any choice of ts such that 

lim \\u s (t s ) - u(t s )\\ =0, (3.10) 

gives a stopping rule: for such t$ one has lim^o II^W — 2/1 1 =0. 
To prove that (3.6) gives such a rule, it is sufficient to check that 

\\u s (t)-u(t)\\<^. (3.11) 
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Let us prove (3.11). Denote w := us — u. Then 

w = -[Bw + ew-p], w(0) = 0, ||p||<<S. (3.12) 

Integrating (3.12), and using the property B > 0, one gets (3.11). 

Alternatively, multiply (3.12) by w, let \\w\\ := g, use B > 0, and get <? < —e(t)g + 
5, ^(0) = 0. Thus, g(t) < 5 exp(— f* eds) f* exp( f Q s edr)ds < ^y. A more precise estimate, 
also used at the end of the proof of Theorem 3.2 below, yields: 

\\u s (t)-u(t)\\ <—?==, (3.11') 

and the corresponding stopping time t$ can be taken as the root of the equation: 

2^/eJtj = 5 b , be (0,1). (3.6') 

Theorem 3.1 is proved. □. 

If the rate of decay of v is known, then a more efficient stopping rule can be derived: 
ts is the minimizer of the problem: 

v(t) + 5[e(t)]- 1 = min. (3.13) 

s 1 

For example, if v(t) < ce a (t), then t$ is the root of the equation e(t) = (^) 1 +° , which 
one gets from (3.13) with v = ce a . 

One can also use a stopping rule based on an a posteriori choice of the stopping time, 
for example, the choice by a discrepancy principle. 

A method, much more efficient numerically than Theorem 3.1, is given below in The- 
orem 4-2. 

For linear equation (3.2) with exact data this method uses (1.4) with 

$ = -{B + e{t)Y l [Bu + e{t)u — q] = —u + (B + e^))" 1 ?, (3.14) 

and for noisy data it uses (1.4) with $>s = — us + (B + eft))^ 1 qs- The linear operator B > 
is monotone, so Theorem 4.2 is applicable. For exact data (1.4) with <I>, defined in (3.14), 
yields: 

u = - U + (B + e{t))- 1 q, u(0) = n , (3.15) 

and (1.6) holds if e(t) > is monotone, continuous, decreasing to as t — > 00. 
Let us formulate the result: 

Theorem 3.2. Assume A), and let B := A* A, q := A*f. Assume e(i) > to be a 
continuous, monotonically decaying to zero function on [0, 00). Then, for any uq G H, 
problem (3.15) has a unique global solution, 3n(oo) = y, Ay = f , and y is the minimal- 
norm solution to (3.1). If fs is given in place of f , \\f — fs\\ < S, then (3.5) holds, with 
us(t) solving (3.15) with q replaced by qs := A* fs, and ts is chosen, for example, as the 
root of (3. & ) (or by a discrepancy principle). 



8 



Proof of Theorem 3.2. One has q = Bz, where Az = f, and the solution to (3.15) 

is 

ft f\\B\\ 

u(t) = u e~* + e~* / e s (B + e{s))~ 1 Bzds := u e~* + / j(\,t)dE x z (3.16) 
Jo Jo 

where 

KA.O-fp^p*, (3-17) 
and E\ is the resolution of the identity of the selfadjoint operator B. One has 

0<j(A,t)<l, lim j(X,t) = 1 A > 0, j(0,*)=0. (3.18) 



t— »oo 



From (3.16)-(3.18) it follows that Eki(oo), u(oo) = z — Pjyz = y, where y is the minimal- 
norm solution to (3.1), N := N(B) = N(A) is the null-space of B and of A, and Pn is 
the orthoprojector onto N in H. This proves the first part of Theorem 3.2. 

To prove the second part, denote w := us — u, g := fs — /, where we dropped the 
dependence on 5 in w and g for brevity. Then w = —w + (B + e(t))~ 1 A*g, w(0) = 0. Thus 

w = e~ l fne s (B + e(s))~ 1 A*qds, so lliyll < 5 e~ l f* — t=c?s < — 4=, where the known 

J0 v w/ 3 M M — JO 2-s/eis) ~ 2 v / 7(t) ' 

estimate (see e.g. [5]) was used: \\(B + e)^ 1 ^*!! < Theorem 3.2 is proved. □ 

4 Nonlinear ill-posed problems with monotone operators 

There is a large literature on the equations (1.1) and (1.4) with monotone operators. In the 
result we present the problem is nonlinear and ill-posed, the new technical tool, Theorem 
4.1, is used, and the stopping rules are discussed. 

Consider (1.4) with monotone F under standard assumptions (1.2) and (1.3), and 

$ = -A-l } (u) [F(u(t)) + e(t)(u(t) - u )] , (4.1) 

where A = A(u) := F'(u), A* is its adjoint, e(t) is the same as in Theorem 3.2, and in 
Theorem 4.2 e(t) is further specified, uq 6 B(uq, R) is an element we can choose to improve 
the numerical performance of the method. If noisy data are given, then, as in Sec. 3, we 
take 

F(u) := B{u) - /, $s = -A-^(u s )[B(u s (t)) - fs + e(t)(u s (t) - u )], 

where H/a — /|| < 5, B is a monotone nonlinear operator, B(y) = /, and us solves (1.7). 

To prove that (1.4) with the above $ has a global solution and (1.6) holds, we use the 
following: 

Theorem 4.1. Let 7(i), cr(t), (3(t) £ C[t , oo) for some real number to. If there exists 
a positive function fi(t) G C 1 [fo,oo) such that 

„< m <!f m .m ]t m <A_ [l(t) -^ ,4.2) 
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where go is the initial condition in (4-3), then a nonnegative solution g to the following 
differential inequality: 



g(t) < -l{t)g(t) + a{t)g\t) + f3(t), g(to) = g , (4.3) 
satisfies the estimate: 

n < n(+\ < - 

for all t € [to i oo)> where 



0<*(i)<^#U-L (4-4) 



< v {t) = ( \ - - + - / f 7 (a) - ^4^) <fa) . (4.5) 

There are several novel features in this result. First, differential equation, which one 
gets from (4.3) by replacing the inequality sign by the equality sign, is a Riccati equation, 
whose solution may blow up in a finite time, in general. Conditions (4.2) guarantee the 
global existence of the solution to this Riccati equation with the initial condition (4.3). 
Secondly, this Riccati differential equation cannot be integrated analytically by separation 
of variables. Thirdly, the coefficient a{t) may grow to infinity as t ^ oo, so that the 
quadratic term does not necessarily has a small coefficient, or the coefficient smaller than 
7(i). Without loss of generality one may assume (3(t) > in Theorem 4.1. The proof of 
Theorem 4.1 is given in [3]. 

The main result of this Section is new. It claims a global convergence in the sense 
that no assumptions on the choice of the initial approximation uo are made. Usually one 
assumes that uq is sufficiently close to the solution of (1.1) in order to prove convergence. 
We take Uq = in Theorem 4.2, because in this theorem uq does not play any role. The 
proof is valid for any choice of u,q, but then the definition of r in Theorem 4.2 is changed. 

Theorem 4.2. If (1.2) and (1.3) hold, uq = 0, R = 3r, where r := \\y\ \ + \ \uo\\, and 
y £ N := {z : F(z) = 0} is the (unique) minimal norm solution to (1.1), then, for any 
choice of uq, problem (1-4) with $ defined in (4-1), = 0, and e(t) = c\{cq + t)~ b with 
some positive constants c±,Co, and b, specified in the proof of Theorem 4-2, has a global 
solution, this solution stays in the ball B(u$, R) and (1.6) holds. If u§(t) solves (1-4) with 

i n place of then there is a ts such that lim^o \ Ws(ts) — y\ \ = 0. 

Proof of Theorem 4.2. Let us sketch the steps of the proof. Let V solve the equation 

F(V) + e(t)V = 0. (4.6) 

Under our assumptions on F, it is well known that: i) (4.6) has a unique solution for 
every t > 0, and ii) sup t>0 ||U|| < ||y||, (cf [3]). If F is Frechet differentiable, then 
V is differentiable, and ||U(t)|| < ||y|||e(i)|/e(t). It is also known that if (1.3) holds, 
then lim^oo \\V(t) — y\\ = 0. We will show that the global solution u to (1.4), with the 
$ from (4.1), does exist, and lim^oo | \u(t) — V(t)\\ = 0. This is done by deriving a 
differential inequality for w := u — V, and by applying Theorem 4.1 to g = \\w\\. Since 
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\\u(t) - y\\ < \\u(t) - V(t)\\ + \\V(t) - y\\, it then follows that (1.6) holds. We also check 
that u(t) G B{uq, R), where R := 3(| \y\ | + ||«o| |), for any choice of uq and a suitable choice 
of e. 

Let us derive the differential inequality for w. One has 

w = -V- A-^{u) [F(u(t)) - F(V(t)) + e(t)w], (4.7) 

and F{u) — F(V) = Aw + K, where ||i^|| < M2g 2 /2, g := \\w\\ and M2 is the constant 
from (1.2). Multiply (4.7) by w, use the monotonicity of F, that is, the property A > 0, 
and the estimate ||y|| < ||?/|||e|/e, and get: 

0.5Mo 2 , , , I e I , A N 

<?< -0 + — ^- + 112/11^' (4-8) 

where M := M 2 . Inequality (4.8) is of the type (4.3): 7 = 1, a = 0.5M/e, /3 = ||y||^. 
Choose 

Clearly /x — ► 00 as t — > 00. Let us check three conditions (4.2). One has = ^. Take 
e = ci(co + i) _b , where Cj > are constants, < b < 1, and choose these constants so 

that ^ < i, for example, ^ = \. Then the first condition (4.2) is satisfied. The second 
condition (4.2) holds if 

8M||y|||e|e" 2 < 1. (4.10) 

One has e(0) = c\Cq b . Choose 

e(0) = 4Mr. (4.11) 

Then 

\e\e^ = bc^(c +t) b -i < bc^c b = ^ = ^L-, (4.12) 

so (4.10) holds. Thus, the second condition (4.2) holds. The last condition (4.2) holds 
because 

2M\\u - Vb| I < 2Mr _ 1 
7(0) ~ 4Mr ~ 2 < 

By Theorem 4.1 one concludes that g = \ \w(t)\\ < |^ — ► when t — > 00, and 

||it(t)-«o|| <fl' + ||V r -« || <#(0)+r<3r. (4.13) 

This estimate implies the global existence of the solution to (1.4), because if u(t) would 
have a finite maximal interval of existence, [0, T), then u(t) could not stay bounded when 
t — > T, which contradicts the boundedness of ||n(t)||, and from (4.13) it follows that 
||-u(t)|| < 4r. We have proved the first part of Theorem 4.2, namely properties (1.6). □ 
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To derive a stopping rule we argue as in Sec. 3. One has: 

\\u S (t) - y\\ < \\u s (t) - V(t)\\ + \\V(t) - y\\. (4.14) 

We have already proved that lim^oo v(t) := lim^oo \ \V(t) — y\\ = 0. The rate of decay of 
v can be arbitrarily slow, in general. Additional assumptions, for example, the source-type 
ones, can be used to estimate the rate of decay of v(t). One derives differential inequality 
(4.3) for gs := ||u<$(i) — ^(£)||, and estimates gs using (4.4). The analog of (4.8) for gs 
contains additional term | on the right-hand side. If ^ < j^j, then conditions (4.2) hold, 
and gs < j^. Let t$ be the root of the equation e 2 (t) = 16M<5. Then lim^o ts = oo, and 
(1.8) holds because \\u s (t s ) - y\\ < v(t s ) + gs, lim^^ g s (t s ) = and lim^oo v(t s ) = 0, 
but the convergence in (1.8) can be slow. See [4] and [5] for the rate of convergence under 
source assumptions. If the rate of decay of v(t) is known, then one chooses t$ as the 
minimizer of the problem, similar to (3.13), 

v(t) + gs(t) = min, (4-15) 

where the minimum is taken over t > for a fixed small 5 > 0. This yields a quasioptimal 
stopping rule. Theorem 4.2 is proved. □ 

In [4] a local convergence result, similar to the first part of Theorem 4.1, was obtained, 
that is, ||iio — y\ | was assumed sufficiently small, and no discussion of noisy data was given. 

Let us give another result: 

Theorem 4.3. Assume that $ = —F(u) — e{t)u, F is monotone, e(t) as in Theorem 
3.2, and (3.3), (1.2) and (1.3) hold. Then (1.6) holds. 

Proof of Theorem 4.3. As in the proof of Theorem 4.2, it is sufficient to prove that 
lim t _ >0O <7(i) = 0, where g,w, and V are the same as in Theorem 4.2, and u solves (1.4) 
with the $ defined in Theorem 4.3. Similarly to the derivation of (4.7), one gets: 

w = -V - [F(u) - F(V) + e{t)w]. (4.16) 

Multiply (4.16) by w, use the monotonicity of F, the estimate ||V"|| < 7^y-||y||, which was 
used also in the proof of Theorem 4.2, and get: 

9<-«t)g+^\\y\\. (4.17) 

This implies 

g(t) <e-ti £ ^ ds [g(0)+ f e^o < x ) dx \^-\\y\\ds}. (4.18) 

Jo e K s ) 

From our assumptions relation (3.3') follows, and (4.18) together with (3.3) and (3.3') 
imply limt_ >00 g(t) = 0. Theorem 4.3 is proved. □ 

Remark 4.4. One can drop assumption (1.2) in Theorem 4.3 and assume only that 
F is a monotone hemicontinuous operator defined on all of H. 
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Claim 4.5: If e(t) = e = const > 0, then lim e ^ ~ v\\ = 0, where u(t) solves 

(1-4) with $ := —F(u) — eu, and t e is any number such that hm e ^oet e = oo. 

Proof of the claim. One has \ \u(t) — y\\ < \\u(t) — V e \ | + | \V e — y\ |, where V e solves 
(4.6) with e(t) = e = const > 0. Under our assumptions on F, equation (4.6) has a 
unique solution, and lim e ^o I IK — 2/1 1 =0. So, to prove the claim, it is sufficient to prove 
that lim e ^o I \u(t e ) — V e \\ = 0, provided that lim e ^oe*e = oo. Let g := \\u(t) — V e \\, and 
w := u(t) — V e . Because V e = 0, one has the equation: w = —[F(u) — F(V e ) + ew]. 
Multiplying this equation by w, and using the monotonicity of F, one gets g < —eg, so 
g{t) < g(0)e~ et . Therefore lim e ^o<7(ie) = 0, provided that lim^o^e = oo. The claim is 
proved. □ 

Remark 4.6. One can prove claims i) and ii), formulated below formula (4.6), using 
DSM version presented in Theorem 8.1 below. 

Claim 4.7: Assume that F is monotone, (1.2) holds, and F(y) = 0. Then claims i) 
and ii), formulated below formula (4-6), hold. 

Proof. First, note that ii) follows from i) easily, because the assumptions F{y) = 0, 
F is monotone, and e > 0, imply, after multiplying F(V) — F(y) + eV = by V — y, the 
inequality (V, V — y) < 0, from which claim ii) follows. Claim i) follows from Theorem 
8.1, proved below. □ 

Claim 4.8: Assume that the operator F is monotone, hemicontinuous, defined on all of 
H , F{y) = 0, y is the minimal norm element of Np := {z : F(z) = 0}, $ = —F(u) — e(t)u, 
e(t) > 0, is monotone, decaying to zero, and (3.3) holds. Then (1.6) holds for the solution 

to (1.4). 

Proof. Existence of the unique global solution to (1.4) under our assumptions is 
known (see e.g. [D]: K.Deimling, Nonlinear functional analysis, Springer Verlag, Berlin, 
1985, p. 99). Let w := u — Vb, g := \ \w\\, where V& solves F(Vb) + bVb = 0, b = const > 0. 
It is shown in the proof of Claim 4.7 that ||14|| < \\y\\, and one can prove (see e.g., 
[2]) that lim^oHH - y\\ = 0. One has \\u(t) - y\\ < \\u{t) - V b \\ + \\V b - y\\. Thus, 
to prove lim^oo \ \u(t) — y\\ = it is sufficient to prove that lim t ^oo g(t) = 0. One has 
w = —[F(u) + e(t)u — F(Vb) — bVb]. Multiply this equation by w, use the monotonicity of 
F and get: g < -e(t)g + \e(t) - b\\\y\\. Denote h(t) := exp( / * e(s)ds). Then, 

9(0 < 5(0)^(0 + ^ _1 (0 f h(s)\e(s) - b\ds\\y\\, V6 = const > 0. (4.19) 

J o 

Clearly lim^^oo g(§)h~ l (^) = 0, because lim^oo hit) = oo. In fact, e _1 < ct + Co, where 
Co := e _1 (0) > 0, and one can choose < c < 1 because of (3.3), so h > (ct + cq)~ , and 
lim^oo e(t)h(t) = oo. Choose b = e(£) and apply L'Hospital's rule to the last term in 
(4.19). L'Hospital's rule is applicable, and one gets: 

hm g(£) = hm -j— -f- = 0, 

because -^7^— is a bounded function and — > as ^ — >• 00. Claim 4.8 is proved. □ 
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The result in claim 4.8 contains the result from [AR] (Alber, Ya. and Ryasantseva, I., 
On regularized evolution equations, Funct. Diff. Eqs, 7, (2000), 177-187.), where addi- 
tional assumptions are made on e(t), global existence of the solution to (1.4) is assumed, 
and the proof contains a gap, because it is not shown that the L'Hospital's rule can be 
applied twice. 

5 Nonlinear ill-posed problems with non-monotone opera- 
tors 

Assume that F(u) := B(u) — f, B is a non-monotone operator, A := F'(u), A := F'(y), 



T := A* A f := A* A, T e := T + el, where I is the identity operator, e is as in Theorem 
3.2 and m < 1, 



and <&s is defined similarly, with fg replacing / and u§ replacing u. 
The main result of this Section is: 

Theorem 5.1. // (1.2) and (1.3) hold, u,u £ B{y,R), y - u = fz, \\z\\ « 1, 
(that is, \\z\\ is sufficiently small), and R is sufficiently small, then problem (1-4) has a 
unique global solution and (1.6) holds. If us(t) solves (1.7), then there exists a t$ such 

that lim^o ||«<s(*<$) ~ u\ \ = °. 

The derivation of the stopping rule, that is, the choice of t$, is based on the ideas 
presented in Sec. 4 (cf [8], [5]). 

Sketch of proof of Theorem 5.1. Proof of Theorem 5.1 consists of the following 
steps. 

First we prove that g := \ \w\\ := \ \u(t) — y\ \ satisfies a differential inequality (4.3), and, 
applying (4.4), conclude that g{t) < /^ _1 (i) —> as t —> oo. A new point in this derivation 
(compared with the one for monotone operators) is the usage of the source assumption 
y - u = fz. 

Secondly, we derive the stopping rule using the ideas from Sec. 4. The source assump- 
tion allows one to get a rate of convergence (see [2] and [5]). Details of the proof are 
technical and are not included. One can see [5] for some proofs. 

Let us sketch the derivation of the differential inequality for g. Write B(u) — f = 
B{u) — B{y) = Aw + K, where | \K\ \ < ^g 2 , and e(u — Uq) = ew + e(y — uq) = ew + eTz. 
Then (5.1) can be written as 



$ := -T-\u)[A*{B{u) - f) + e(u - u )], e = e(t) > 0, 



(5.1) 



$ = -w -T- l A*K - eT^fz, e: 



e(t). 



(5.2) 



Multiplying (1.4), with $ defined in (5.2), by w, one gets: 



99 < -9 2 + -^\\T-^A*\\g 3 + e(t)\\T^f\\\\z\\g. 



(5.3) 
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Since g > 0, one obtains: 

g <-g + J^g* + e{t)\\T~ l f\\\\z\\, (5.4) 
where the estimate IIT" 1 ^*!! < was used. Clearly, 

||r e -1 f|| < ||(T e _1 -f-^fw + \\f- 1 f\\, Wf^fw < i, eHT" 1 !! < 1, 

and 

T -i _ f-i = T -i( A * A _ A*A)f~ l . 

One has: 

\\A*A- A*A\\ <2M 2 M ig , \\z\\ « 1. 

Let 2M1M2II2H < \. This is possible since ||z|| << 1. Using the above estimates, one 
transforms (5.4) into the following inequality: 

5<-^+-^=5 2 + IMk. (5-5) 
^ 4y / e(t) 

Now, apply Theorem 4.1 to (5.5), choosing 

2M 2 \e\ 1 2M 2 \\u -y\\ 

H = —^, — < -, 16M 2 ||z||Ve(0) < 1, and '-== < 1. 

ye e I V e (°) 

Then conditions (4.2) are satisfied, and Theorem 4.1 yields the estimate: 



9{t) < 2M 2 - 
This is the main part of the proof of Theorem 5.1. □ 



6 Nonlinear ill-posed problems: avoiding inverting of oper- 
ators in the Newton-type continuous schemes 

In the Newton-type methods for solving well-posed nonlinear problems, for example, in the 
continuous Newton method (1.4) with <I> = — [F'(u)]~ 1 F(u), the difficult and expensive 
part of the solution is inverting the operator F'(u). In this section we give a method to 
avoid inverting of this operator. This is especially important in the ill-posed problems, 
where one has to invert some regularized versions of F' , and to face more difficulties than 
in the well-posed problems. 

Consider problem (1.1) and assume (1.2), (1.3) and (1.9). Thus, we discuss our method 
in the simplest well-posed case. 

Replace (1.4) by the following Cauchy problem (dynamical system): 
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ii = -QF, u(0) = u , (6.1) 
Q = -TQ + A*, Q(0) = Q , (6.2) 

where A := F'(u), T := A* A, and Q = Q(t) is a bounded operator in H. 

First let us state our new technical tool: an operator version of the Gronwall inequality 
(cf [9]). 

Theorem 6.1. Let 

Q = -T(t)Q(t) + G(t), Q(0) = Q , (6.3) 

where T(t), G(t), and Q(t) are linear bounded operators on a real Hilbert space H. If there 
exists e(t) > such that 

{T(t)h,h)>e(t)\\h\\ 2 VheH, (6.4) 



then 



\\Q{t)\\ < e-/o^[||Q(0)|| + f \\G(s)\\efo <*) d * ds}. (6.5) 

Jo 



Let us turn now to a proof of Theorem 6.2, formulated at the end of this Section. This 
theorem is the main result of Section 6. 

Applying (6.5) to (6.2), and using (1.2) and (1.9), which implies 

(T(t)h, h) > c\\h\\ 2 Vh£H, c = const > 0, (6.6) 

one gets: 

\\Q(t)\\ <e" c *[||Q(0)||+ f M ie cs ds] < l\\Qo\\ + M 1 c- 1 } := Cl , (6.7) 

J o 

as long as u(t) € B(uq, R). 

Let u(t) — y := w, \\w\\ := g, A := F'(y). Since F(y) = 0, one has F(u) = Aw + K, 
where \ \K\ \ < 0.5M2<7 2 := cog 2 , and M2 is the constant from (1.2). Rewrite (6.1) as 

w = -Q[Aw + K]. (6.8) 

Let A := I - QA. Multiply (6.8) by w and get 

99 < —g 2 + (Aw;, w) + cog 3 , cq = const > 0. (6.9) 

We prove below that 

sup || A|| < A < 1. (6.10) 

t>o 

From (6.9) and (6.10) one gets the following differential inequality: 

9<-19 + c g 2 , 0< 7 <1, 7 :=1-A, (6.11) 
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which implies: 

g{t) <re-*, r := g(Q)[l ~ 9{Q)<*r\ (6-12) 

provided that 

g(0)c < 1. (6.13) 

Inequality (6.13) holds if uq is sumciently close to y. 

From (6.12) and (6.11) it folows that u(oo) = y. Thus, (1.6) holds. 
The trajectory u(t) G B(uq, R), Vf > 0, provided that 

f' 00 f°° cnr 2 

/ \\u\\dt= \\w\\dt < r + -2— < R. (6.14) 
Jo Jo 2 7 

This inequality holds if no is sumciently close to y, that is, r is sumciently small. 
To complete the argument, let us prove (6.10). One has: 

A = -QA = -TA + A*(A-A). (6.15) 

One has \ \A — A\\ < M 2 g. Using (6.12) and Theorem 6.1, one gets 

||A|| < e- ct [||A || +rM 1 M 2 f e^'ds]. (6.16) 

Jo 

Thus, 

||A|| < ||A || + Cr := A, C := MiM 2 sup- — — . (6.17) 

00 c-7 

If no is sumciently close to y and Qq is sumciently close to A~ 1 , then A > can be made 
arbitrarily small. We have proved: 

Theorem 6.2. If (1.2), (1.3) and (1.9) hold, Qq and uq are sufficiently close to A^ 1 
andy, respectively, then problem (6.1) -(6. 2) has a unique global solution, (1.6) holds, and 
u(t) converges to y, which solves (1.1), exponentially fast. 

In [9] a generalization of Theorem 6.2 is given for ill-posed problems. 

7 Iterative schemes 

In this section we present a method for constructing convergent iterative schemes for a wide 
class of well-posed equations (1.1). Some methods for constructing convergent iterative 
schemes for a wide class of ill-posed problems are given in [3]. There is an enormous 
literature on iterative methods. 

Consider a discretization scheme for solving (1.4) with <I> = 3>(n), so that we assume 
no explicit time dependence in $: 

n n+ i = u n + h<&(u n ), uq = no, h = const > 0. (7.1) 
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One of our results from [3], concerning the well-posed equations (1.1) is Theorem 7.1, 
formulated below. Its proof is shorter and simpler than in [3]. 

Theorem 7.1. Assume (1.2), (1.3), (1.9), (2.1)-(2.4) with a = 2, g 1 =c 1 = const > 
0, 92 = C2 = const > 0, \\<f>'(u)\\ < L\, for u € B(y,R). Then, if h > is sufficiently 
small, and uq is sufficiently close to y, then (7.1) produces a sequence u n for which 

IK -y\\ < Re~ chn , \\F(u n )\\ < ||F ||e- chn , (7.2) 

where R := 2&M t F = F(u ), c = const > 0, c < c\. 

Proof of Theorem 7.1. The proof is by induction. For n = estimates (7.2) 
are clear. Assuming these estimates for j < n, let us prove them for j = n + 1. Let 
F n := F(u n ), and let w n+ \{t) solve problem (1.4) on the interval (t n ,t n+ i), t n := nh, with 
w(t n ) = u n . By (2.5) (with G = c 2 e~ Clt ) and (7.2) one gets: 

\\w n+1 (t) -y\\< ^||F n || e - Clt < Re- cnh ^\ t n < t < t n+1 . (7.3) 
^1 



One has: 
and 



||u n +i-y|| < \ \u n +i ~ w n+ i(t n+1 )\\ + \\w n+ i(t n+1 ) - y\\, (7.4) 

rtn+l 

\\u n+ \ - W n+ l(t n+ l )|| < / \Mu n ) - $(w n+1 (s))\\ds 

J t n 

< L lC2 h j " +1 \\F(w n+1 (t))\\dt < L lCl h 2 Re- cnh , (7.5) 
where we have used the formula R := C2 ^ F °^ and the estimate: 

Cl ' 

\\F(w n+1 (t))\\ < ||F n ||e- cl (*-*") < ||Fo||e- cnft - cl ( t - t "). (7.6) 
From (7.3)-(7.6) it follows that: 

| |u n+ i -y\\ < Re- cnh (e- Clh + c^h 2 ) < Re-^ 1 ^, (7.7) 

provided that 

e~ Clh + c 1 L 1 h 2 <e^ ch . (7.8) 

Inequality (7.8) holds if h is sufficiently small and c < c\. So, the first inequality (7.2), 
with n + 1 in place of n, is proved if h is sufficiently small and c < c\. 
Now 

H^K+i)!! < ||F(n n+1 ) - F(«; n+ i(t))|| + \\F(w n+1 (t))\\, t n <t< t n+1 . (7.9) 
Using (1.2) and (7.5), one gets: 
\\F(u n+1 ) - F(w n+1 (t n+1 ))\\ < M 1 \\u n+1 - ^ n+ i(t n+ i)|| < M x c 2 L x h 2 \\Fv\\e- cnh . (7.10) 
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From (7.9) and (7.10) it follows that: 

\\F(u n+1 )\\ < ||F ||e- cn/l (e- Cl/l + M 1 c 2 L 1 h 2 ) < \\F \\e- c ^ h , (7.11) 

provided that 

e -cih + Mic 2 Li/i 2 < e~ ch . (7.12) 

Inequality (7.12) holds if h is sufficiently small and c < c\. So, the second inequality (7.2) 
with n + 1 in place of n is proved if h is sufficiently small and c < c\. Theorem 7.1 is 
proved. □ 

In the well-posed case, if F(y) = 0, the discrete Newton's method 

u n+ i =u n - [F' (un)] -1 F(u n ) , u = u(0), 

converges superexponentially if uq is sufficiently close to y. Indeed, if v n := u n — y, then 
Vn+i =v n - [F'(u n )]~ 1 [F'(u n )v n + K] where ||if|| < ^||f„|| 2 . Thus, g n := \\v n \\ satisfies 
the inequality: g n+ \ < qg„, where q := mi 2 M2 . Therefore g n < g 2 "" 1 ^", and if < qgo < 1, 
then the method converges superexponentially. 

If one uses the iterative method u n+ \ = u n — h[F'(u n )]~ 1 F(u n ), with h ^ 1, then, in 
the well-posed case, assuming that this method converges, it converges exponentially, that 
is, slower than in the case h = 1. 

The continuous analog of the above method 

u = -a[F'(u)] _1 F(n), u(0) = u , 

where a = const > 0, converges at the rate 0(e~ at ). Indeed, if g(t) := ||F(k(£))||, then 
gg = -ag 2 , so g(t) = g e~ at , \\u\ \ < am x g§e~ at . Thus 

\\u(t) - n(oo)|| < mig e~ at , and F(u(oo)) = 0. 

In the continuous case one does not have super exponential convergence no matter what 
a > is (see [11]). 

8 A spectral assumption 

In this section we introduce the spectral assumption which allows one to treat some non- 
linear non-monotone operators. 

Assumption S: The set {r, (p : it — ipo < ip < ir + ipo, ifo > 0, < r < vq}, where ipo 
and ro are arbitrarily small, fixed numbers, consists of the regular points of the operator 
A := F'(u) for all u G B(u ,R). 

Assumption S implies the estimate: 

\\(F'(u) + e) _1 || < — - — , e < r (l -sinp ), e = const > 0, (8.1) 
e sin ipo 
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because — z) 1 || < ^i st ^ s (A)) ' wnere s (^) i s the spectrum of a linear operator A, and 
dist(z, s(A)) is the distance from a point z of a complex plane to the spectrum. In our 
case, z = — e, and dist(z, s(A)) = esin^o, if e < ro(l — sin</?o). 

Theorem 8.1. If (1.2) and (8.1) hold, andO < e < ro(l — sin tpo), then problem (4-6), 
with e(t) = e = const > ; is solvable, problem (1-4), with & defined in (4-1) and uq = 0, 
has a unique global solution, 3u(oo), and F(n(oo)) + en(oo) = 0. Every solution to the 
equation F(V) + eV = is isolated. 

Proof of Theorem 8.1. Let g = g(t) := \\F(u(t)) + eu(t)\\, where u = u(t) solves 
locally (1.4), where $ is defined in (4.1) and uq = 0. Then: 

gg = -((F'(u) + e)(F'(u) + e)" 1 ^) + eu), F(u) + eu) = -g 2 , (8.2) 



so 



9 = 9oe~ t , go :=S(0); ||u|| < — ^— e"*. (8.3) 

e sin 



Thus, 



|Kt)-M(oo)|| < 90 e~\ I km -Moll < 90 , F(u( oo)) + eu(oo) = 0. (8.4) 
esm<p esmipo 

Therefore equation 

F(V) + eV = 0, e = const > 0, (8.5) 
has a solution in -B(un, R), where R = — ^ — . 

v u ' / ' e sin ipo 

Every solution to equation (8.5) is isolated. Indeed, if F(W)+eW = 0, and ip := V—W, 
then F(V) - F(W) + eip = 0, so [^(V) + e]ip + X = 0, where ||K|| < ^||V>|| 2 - Thus, 
using (8.1), one gets \ \^\\ > 2e s ^ po ■ Consequently, if ||V>|| is sufficiently small, then ip = 0. 
Theorem 8.1 is proved. □ 
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